Methods and systems for detecting frame tears

ABSTRACT

Methods and systems for detecting frame tears are described. As one example, a mobile device may include at least one camera, a sensor, a co-processor, and an application processor. The co-processor is configured to generate a digital image including image data from the at least one camera and sensor data from the sensor. The co-processor is further configured to embed a frame identifier corresponding to the digital image at least two corner pixels of the digital image. The application processor is configured to receive the digital image from the co-processor, determine a first value embedded in a first corner pixel of the digital image, and determined a second value embedded in a second corner pixel of the digital image. The application processor is also configured to provide an output indicative of a validity of the digital image based on a comparison between the first value and the second value.

BACKGROUND

In addition to having advanced computing and connectivity capabilitiesto facilitate high-speed data communication, many modern mobile devicesinclude a variety of sensors. For example, mobile devices, such assmartphones, tablets, and wearable computing devices, are often equippedwith sensors for imaging and positioning. A few examples of sensors thatmay be found in a mobile device include accelerometers, gyroscopes,magnetometers, barometers, global positioning system (GPS) receivers,microphones, cameras, Wi-Fi sensors, Bluetooth sensors, temperaturesensors, and pressure sensors, among other types of sensors.

The wide variety of available sensors enables mobile devices to performvarious functions and provide various user experiences. As one example,a mobile device may use imaging and/or positioning data to determine atrajectory of the mobile device as a user moves the mobile devicethrough an environment. As another example, a mobile device may useimaging and/or positioning data to generate a 2D or 3D map of anenvironment, or determine a location of a mobile device within a 2D or3D map of an environment. As a further example, a mobile device may useimaging and/or positioning data to facilitate augmented realityapplications. Other examples also exist.

SUMMARY

Within examples, a co-processor of the mobile device may be configuredto provide data from two or more sensors to an application processorwithin a single data structure. For instance, the co-processor may beconfigured to generate a digital image that includes image data acquiredusing a camera of the mobile device as well as sensor data from anothersensor of the mobile device. To include the sensor data within thedigital image, the co-processor may be configured to embed the sensordata within one or more pixels of the digital image. The digital imagecan then be transmitted to the application processor over a camera businterface in order to quickly transmit a large amount of data. Moreover,multiple digital images can be provided in sequence, as new image dataand sensor data is acquired by the mobile device.

Disclosed herein are methods and systems that enable testing for errorsresulting from the generation, transmission, or reception of such adigital image. As described below, example methods may involve embeddingone or more frame identifiers within the digital image at predeterminedpixel positions. The embedded frame identifiers may enable theapplication processor to test for errors in the digital image. Forexample, the described methods and systems may allow the applicationprocessor to identify whether a particular digital image includes datathat was intended to be included within a prior or subsequent digitalimage but was improperly included within the digital image.

In one example aspect, a method is provided. The method includesreceiving, by a processor of a mobile device, image data from at leastone camera of the mobile device and sensor data from at least one sensorof a plurality of sensors of the mobile device. The method also includesgenerating, by the processor, a digital image that includes at least theimage data and the sensor data. The sensor data may be embedded inpixels of the digital image. The method further includes embedding, bythe processor, a frame identifier corresponding to the digital image inat least two corner pixels of respective corners of the digital image.Additionally, the method includes providing, by the processor, thedigital image to an application processor of the mobile device.

In another example aspect, a mobile device that includes at least onecamera, a sensor, a co-processor, and an application processor isprovided. The co-processor is configured to generate a digital imagethat includes image data from the at least one camera and sensor datafrom the sensor. The sensor data is embedded in pixels of the digitalimage. The co-processor is further configured to embed a frameidentifier corresponding to the digital image in at least two cornerpixels of respective corners of the digital image. The applicationprocessor is configured to receive the digital image from theco-processor, determine a first value embedded in a first corner pixelof the at least two corner pixels, and determine a second value embeddedin a second corner pixel of the at least two corner pixels. Theapplication processor is also configured to provide an output indicativeof a validity of the digital image based on a comparison between thefirst value and the second value.

In still another example aspect, a method is provided. The methodincludes receiving, by an application processor of a mobile device andfrom a co-processor of the mobile device, a digital image that includesimage data from at least one camera of the mobile device and sensor datafrom at least one sensor of the mobile device. The sensor data isembedded in pixels of the digital image. The digital image also includesa frame identifier embedded in at least two corner pixels of respectivecorners of the digital image. The method also includes determining, bythe application processor, a first value embedded in a first cornerpixel of the at least two corner pixels and a second value embedded in asecond corner pixel of the at least two corner pixels. The methodfurther includes providing an output indicative of a validity of thedigital image based on a comparison between the first value and thesecond value.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the figures and the followingdetailed description.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates an example computing device.

FIG. 2 illustrates another example computing device.

FIGS. 3A-3B are conceptual illustrations of an example computing device.

FIG. 4 is a block diagram of an example method for generating a digitalimage.

FIG. 5 is a conceptual illustration of an example digital image format.

FIG. 6 is a conceptual illustration of example sensor data formats.

FIG. 7 is another conceptual illustration of an example digital imageformat.

FIG. 8 is another conceptual illustration of an example digital imageformat.

FIG. 9 is a block diagram of an example method for testing a digitalimage.

FIG. 10 is a block diagram of additional or optional functions that maybe performed in conjunction with the example method of FIG. 9.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying figures, which form a part hereof. In the figures, similarsymbols typically identify similar components, unless context dictatesotherwise. The illustrative embodiments described in the detaileddescription, figures, and claims are not meant to be limiting. Otherembodiments may be utilized, and other changes may be made, withoutdeparting from the scope of the subject matter presented herein. It willbe readily understood that the aspects of the present disclosure, asgenerally described herein, and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations, all of which are explicitlycontemplated herein.

In examples in which a mobile device relies on data from two or moresensors to perform a particular function (e.g., trajectorydetermination, map generation, etc.), it can be beneficial to providedata from each of the sensors to an application processor within asingle data structure. Providing data from each of the sensors within asingle data structure may help ensure that data from each of the sensorscorresponds to data collected during the same instance in time or a sametime period.

If the data from one or more of the sensors is an image, one solution isto generate a digital image that includes both the image and the sensordata from another sensor. For instance, a co-processor of the mobiledevice may be configured to embed the sensor data within one or morepixels of the digital image. The digital image can then be provided tothe application processor using a camera bus interface. Furthermore, ifimage data and sensor data are repeatedly being acquired (e.g., at afrequency of 60 Hz, or at other rates), multiple digital images thatinclude new image data and sensor data can be generated and provided insequence to the application processor.

When multiple digital images are being generated by the co-processor andprovided to the application processor at a high frame rate, there is apossibility of frame tearing occurring. A frame tear may occur when dataintended to be included within a first digital frame is mistakenlyincluded within a second digital frame (e.g., a prior or subsequentdigital frame). Similarly, frame tearing may unintentionally occur ifone or more data packets are dropped during the generation ortransmission of a digital image.

Disclosed herein are methods and systems that enable testing for frametears, or other possible errors, resulting from the generation,transmission, or reception of a digital image that includes data fromtwo or more sensors of the mobile device. In one example, a co-processorof the mobile device may be configured to generate a digital image thatincludes image data from at least one camera of the mobile device andsensor data from at least one sensor of the mobile device. For instance,the sensor data may be accelerometer data that is embedded within pixelsof the digital image. Furthermore, the co-processor may be configured toembed a frame identifier corresponding to the digital image in at leasttwo corner pixels of respective corners of the digital image.

As an example, the frame identifier may be a frame number correspondingto the digital image. In some instances, the frame identifier may beembedded in a top-left corner pixel and a bottom-right corner pixel ofthe digital image. In other instances, the frame identifier may beembedded in a corner pixel of each of the corners of the digital image.As described herein, a corner pixel need not be the precise corner pixelof the digital image. Rather, a corner pixel may refer to a pixel thatis within a threshold number of columns and a threshold number of rowsfrom a corner of the digital image. For instance, a corner pixel may bedefined as any pixel that is located within five rows and within fivecolumns of a corner of the digital image, such that a pixel located inpixel position of row 3, column 3 of a digital image may be considered acorner pixel.

Additionally, in some examples, the digital image may be divided intoone or more segments or regions, and the co-processor may be configuredto embed frame identifiers within respective regions of the digitalimage. For instance, the digital image may include a first segment ofpixels corresponding to image data from a camera of the mobile deviceand second segment of pixels corresponding to sensor data from one ormore sensors of the mobile device. The co-processor may then beconfigured to embed a frame identifier within one or more corner pixelsof the first segment of pixels and one or more corner pixels of thesecond segment of pixels.

In some instances, the frame identifier embedded in the corner pixels ofthe first segment and second segment (“segment frame identifier”) may bethe same as the frame identifier identified in the corner pixels of thedigital image (“global frame identifier”). In other instances, thesegment frame identifier may be different from the global frameidentifier. Furthermore, the segment frame identifier embedded withinthe first segment of pixels may be the same or different from thesegment frame identifier embedded within the second segment of pixels.

Additionally, within examples, an application processor that receives agenerated digital image from the co-processor may be configured to testthe received digital image for errors using the embedded global frameidentifiers and/or segment frame identifiers. The application processormay be configured to identify frame identifiers embedded withinpredetermined pixel positions, and compare the frame identifiers to oneanother. For instance, the application processor may be configured todetermine a first value embedded in a first corner pixel of the digitalimage, and determine a second value embedded in a second corner pixel ofthe digital image. Based on a comparison between the first value and thesecond value, the application processor may provide an output indicativeof a validity of the digital frame. For example, the applicationprocessor may provide an output indicating that the digital frame isvalid in response to determining that the first value is equal to thesecond value.

Similarly, in some examples, the application processor may be configuredto identify the value of two or more segment frame identifiers embeddedwithin a particular segment of pixels of the digital image. Forinstance, the application processor may be configured to compare a firstvalue embedded within a first corner pixel of a segment of pixels and asecond value embedded within a second corner pixel of the segment ofpixels. The application processor may then provide an output indicativeof a validity of the digital image. If the application processor isconfigured to test the digital image using global frame identifiers aswell as the segment frame identifiers, the application processor mayprovide an output indicating that the digital frame is valid if each ofthe global frame identifiers are equal and each of the segment frameidentifiers are equal.

Additionally or alternatively, the application processor may provideanother output indicating a validity of the segment of pixels,irrespective of the validity of the digital image. For example, if eachof the segment frame identifiers embedded within the segment of pixelsare equal, the application processor may provide an output indicatingthat the segment of pixels is valid.

Additional example methods as well as example devices (e.g., mobile orotherwise) are described hereinafter with reference to the accompanyingfigures.

Referring now to the figures, FIG. 1 illustrates an example computingdevice 100. In some examples, components illustrated in FIG. 1 may bedistributed across multiple computing devices. However, for the sake ofexample, the components are shown and described as part of one examplecomputing device 100. The computing device 100 may be or include amobile device (such as a mobile phone), desktop computer, laptopcomputer, email/messaging device, tablet computer, or similar devicethat may be configured to perform the functions described herein.Generally, the computing device 100 may be any type of computing deviceor transmitter that is configured to transmit data or receive data inaccordance with methods and functions described herein.

The computing device 100 may include an interface 102, a wirelesscommunication component 104, a cellular radio communication component106, a global positioning system (GPS) receiver 108, sensor(s) 110, datastorage 112, and processor(s) 114. Components illustrated in FIG. 1 maybe linked together by a communication link 116. The computing device 100may also include hardware to enable communication within the computingdevice 100 and between the computing device 100 and other computingdevices (not shown), such as a server entity. The hardware may includetransmitters, receivers, and antennas, for example.

The interface 102 may be configured to allow the computing device 100 tocommunicate with other computing devices (not shown), such as a server.Thus, the interface 102 may be configured to receive input data from oneor more computing devices, and may also be configured to send outputdata to the one or more computing devices. The interface 102 may beconfigured to function according to a wired or wireless communicationprotocol. In some examples, the interface 102 may include buttons, akeyboard, a touchscreen, speaker(s) 118, microphone(s) 120, and/or anyother elements for receiving inputs, as well as one or more displays,and/or any other elements for communicating outputs.

The wireless communication component 104 may be a communicationinterface that is configured to facilitate wireless data communicationfor the computing device 100 according to one or more wirelesscommunication standards. For example, the wireless communicationcomponent 104 may include a Wi-Fi communication component that isconfigured to facilitate wireless data communication according to one ormore IEEE 802.11 standards. As another example, the wirelesscommunication component 104 may include a Bluetooth communicationcomponent that is configured to facilitate wireless data communicationaccording to one or more Bluetooth standards. Other examples are alsopossible.

The cellular radio communication component 106 may be a communicationinterface that is configured to facilitate wireless communication (voiceand/or data) with a cellular wireless base station to provide mobileconnectivity to a network. The cellular radio communication component106 may be configured to connect to a base station of a cell in whichthe computing device 100 is located, for example.

The GPS receiver 108 may be configured to estimate a location of thecomputing device 100 by precisely timing signals sent by GPS satellites.

The sensor(s) 110 may include one or more sensors, or may represent oneor more sensors included within the computing device 100. Examplesensors include an accelerometer, gyroscope, pedometer, light sensor,microphone, camera(s), infrared flash, barometer, magnetometer, Wi-Fi,near field communication (NFC), Bluetooth, projector, depth sensor,temperature sensor, or other location and/or context-aware sensors.

The data storage 112 may store program logic 122 that can be accessedand executed by the processor(s) 114. The data storage 112 may alsostore data collected by the sensor(s) 110, or data collected by any ofthe wireless communication component 104, the cellular radiocommunication component 106, and the GPS receiver 108.

The processor(s) 114 may be configured to receive data collected by anyof sensor(s) 110 and perform any number of functions based on the data.As an example, the processor(s) 114 may be configured to determine oneor more geographical location estimates of the computing device 100using one or more location-determination components, such as thewireless communication component 104, the cellular radio communicationcomponent 106, or the GPS receiver 108. The processor(s) 114 may use alocation-determination algorithm to determine a location of thecomputing device 100 based on a presence and/or location of one or moreknown wireless access points within a wireless range of the computingdevice 100. In one example, the wireless location component 104 maydetermine the identity of one or more wireless access points (e.g., aMAC address) and measure an intensity of signals received (e.g.,received signal strength indication) from each of the one or morewireless access points. The received signal strength indication (RSSI)from each unique wireless access point may be used to determine adistance from each wireless access point. The distances may then becompared to a database that stores information regarding where eachunique wireless access point is located. Based on the distance from eachwireless access point, and the known location of each of the wirelessaccess points, a location estimate of the computing device 100 may bedetermined.

In another instance, the processor(s) 114 may use alocation-determination algorithm to determine a location of thecomputing device 100 based on nearby cellular base stations. Forexample, the cellular radio communication component 106 may beconfigured to identify a cell from which the computing device 100 isreceiving, or last received, signal from a cellular network. Thecellular radio communication component 106 may also be configured tomeasure a round trip time (RTT) to a base station providing the signal,and combine this information with the identified cell to determine alocation estimate. In another example, the cellular communicationcomponent 106 may be configured to use observed time difference ofarrival (OTDOA) from three or more base stations to estimate thelocation of the computing device 100.

In some implementations, the computing device 100 may include a deviceplatform (not shown), which may be configured as a multi-layered Linuxplatform. The device platform may include different applications and anapplication framework, as well as various kernels, libraries, andruntime entities. In other examples, other formats or operating systemsmay operate the computing g device 100 as well.

The communication link 116 is illustrated as a wired connection;however, wireless connections may also be used. For example, thecommunication link 116 may be a wired serial bus such as a universalserial bus or a parallel bus, or a wireless connection using, e.g.,short-range wireless radio technology, or communication protocolsdescribed in IEEE 802.11 (including any IEEE 802.11 revisions), amongother possibilities.

The computing device 100 may include more or fewer components. Further,example methods described herein may be performed individually bycomponents of the computing device 100, or in combination by one or allof the components of the computing device 100.

FIG. 2 illustrates another example computing device 200. The computingdevice 200 in FIG. 2 may be representative of a portion of the computingdevice 100 shown in FIG. 1. In FIG. 2, the computing device 200 is shownto include a number of sensors such as an inertial measurement unit(IMU) 202 including a gyroscope 204 and an accelerometer 206, a globalshutter (GS) camera 208, a rolling shutter (RS) camera 210, a frontfacing camera 212, an infrared (IR) flash 214, a barometer 216, amagnetometer 218, a GPS receiver 220, a Wi-Fi/NFC/Bluetooth sensor 222,a projector 224, and a temperature sensor 226, each of which outputs toa co-processor 230. Additionally, the computing device 200 is shown toinclude a depth processor 228 that receives input from and outputs tothe co-processor 230. And the co-processor 230 receives input from andoutputs to an application processor 232. The computing device 200 mayfurther include a second IMU 234 that outputs directly to theapplication processor 232.

The IMU 202 may be configured to determine a velocity, orientation, andgravitational forces of the computing device 200 based on outputs of thegyroscope 204 and the accelerometer 206.

The GS camera 208 may be configured on the computing device 200 to be arear facing camera, so as to face away from a front of the computingdevice 200. The GS camera 208 may be configured to read outputs of allpixels of the camera 208 simultaneously. The GS camera 208 may beconfigured to have about a 120-170 degree field of view, such as a fisheye sensor, for wide-angle viewing.

The RS camera 210 may be configured to read outputs of pixels from a topof the pixel display to a bottom of the pixel display. As one example,the RS camera 210 may be a red/green/blue (RGB) infrared (IR) 4megapixel image sensor, although other sensors are possible as well. TheRS camera 210 may have a fast exposure so as to operate with a minimumreadout time of about 5.5 ms, for example. Like the GS camera 208, theRS camera 210 may be a rear facing camera.

The camera 212 may be an additional camera in the computing device 200that is configured as a front facing camera, or in a direction facingopposite of the GS camera 208 and the RS camera 210. The camera 212 maybe a wide angle camera, and may have about a 120-170 degree field ofview for wide angle viewing, for example.

The IR flash 214 may provide a light source for the computing device200, and may be configured to output light in a direction toward a rearof the computing device 200 so as to provide light for the GS camera 208and RS camera 210, for example. In some examples, the IR flash 214 maybe configured to flash at a low duty cycle, such as 5 Hz, or in anon-continuous manner as directed by the co-processor 230 or applicationprocessor 232. The IR flash 214 may include an LED light sourceconfigured for use in mobile devices, for example.

FIGS. 3A-3B are conceptual illustrations of a computing device 300 thatshow a configuration of some of the sensors on the computing device 300.In FIGS. 3A-3B, the computing device 300 is shown as a mobile phone. Thecomputing device 300 may be similar to either of computing device 100 inFIG. 1 or computing device 200 in FIG. 2. FIG. 3A illustrates a front ofthe computing device 300 in which a display 302 is provided, along witha front facing camera 304, and a P/L sensor opening 306 (e.g., aproximity or light sensor). The front facing camera 304 may be thecamera 212 as described in FIG. 2.

FIG. 3B illustrates a back 308 of the computing device 300 in which arear camera 310 and another rear camera 314 are provided. The rearcamera 310 may be the RS camera 210 and the rear camera 312 may be theGS camera 208, as described in the computing device 200 in FIG. 2. Theback 308 of the computing device 300 also includes an IR flash 314,which may be the IR flash 214 or the projector 224 as described in thecomputing device 200 in FIG. 2. In one example, the IR flash 214 and theprojector 224 may be one in the same. For instance, a single IR flashmay be used to perform the functions of the IR flash 214 and theprojector 224. In another example, the computing device 300 may includea second flash (e.g., an LED flash) located near the rear camera 310(not shown). A configuration and placement of the sensors may be helpfulto provide desired functionality of the computing device 300, forexample, however other configurations are possible as well.

Referring back to FIG. 2, the barometer 216 may include a pressuresensor, and may be configured to determine air pressures and altitudechanges.

The magnetometer 218 may be configured to provide roll, yaw, and pitchmeasurements of the computing device 200, and can be configured tooperate as an internal compass, for example. In some examples, themagnetometer 218 may be a component of the IMU 202 (not shown).

The GPS receiver 220 may be similar to the GPS receiver 108 described inthe computing device 100 of FIG. 1. In further examples, the GPS 220 mayalso output timing signals as received from GPS satellites or othernetwork entities. Such timing signals may be used to synchronizecollected data from sensors across multiple devices that include thesame satellite timestamps.

The Wi-Fi/NFC/Bluetooth sensor 222 may include wireless communicationcomponents configured to operate according to Wi-Fi and Bluetoothstandards, as discussed above with the computing device 100 of FIG. 1,and according to NFC standards to establish wireless communication withanother device via contact or coming into close proximity with the otherdevice.

The projector 224 may be or include a structured light projector thathas a laser with a pattern generator to produce a dot pattern in anenvironment. The projector 224 may be configured to operate inconjunction with the RS camera 210 to recover information regardingdepth of objects in the environment, such as three-dimensional (3D)characteristics of the objects. For example, the RS camera 210 may be anRGB-IR camera that is configured to capture one or more images of thedot pattern and provide image data to the depth processor 228. The depthprocessor 228 may then be configured to determine distances to andshapes of objects based on the projected dot pattern. By way of example,the depth processor 228 may be configured to cause the projector 224 toproduce a dot pattern and cause the RS camera 210 to capture an image ofthe dot pattern. The depth processor may then process the image of thedot pattern, use various algorithms to triangulate and extract 3D data,and output a depth image to the co-processor 230.

The temperature sensor 226 may be configured to measure a temperature ortemperature gradient, such as a change in temperature, for example, ofan ambient environment of the computing device 200.

The co-processor 230 may be configured to control all sensors on thecomputing device 200. In examples, the co-processor 230 may controlexposure times of any of cameras 208, 210, and 212 to match the IR flash214, control the projector 224 pulse sync, duration, and intensity, andin general, control data capture or collection times of the sensors. Theco-processor 230 may also be configured to process data from any of thesensors into an appropriate format for the application processor 232. Insome examples, the co-processor 230 merges all data from any of thesensors that corresponds to a same timestamp or data collection time (ortime period) into a single data structure to be provided to theapplication processor 232. The co-processor 230 may also be configuredto perform other functions, as described below.

The application processor 232 may be configured to control otherfunctionality of the computing device 200, such as to control thecomputing device 200 to operate according to an operating system or anynumber of software applications stored on the computing device 200. Theapplication processor 232 may use the data collected by the sensors andreceived from the co-processor to perform any number of types offunctionality. The application processor 232 may receive outputs of theco-processor 230, and in some examples, the application processor 232may receive raw data outputs from other sensors as well, including theGS camera 208 and the RS camera 210. The application processor 232 mayalso be configured to perform other functions, as described below.

The second IMU 234 may output collected data directly to the applicationprocessor 232, which may be received by the application processor 232and used to trigger other sensors to begin collecting data. As anexample, outputs of the second IMU 234 may be indicative of motion ofthe computing device 200, and when the computing device 200 is inmotion, it may be desired to collect image data, GPS data, etc. Thus,the application processor 232 can trigger other sensors throughcommunication signaling on common buses to collect data at the times atwhich the outputs of the IMU 234 indicate motion.

The computing device 200 shown in FIG. 2 may include a number ofcommunication buses between each of the sensors and processors. Forexample, the co-processor 230 may communicate with each of the IMU 202,the GS camera 208, and the RS camera 212 over an inter-integratedcircuit (I2C) bus that includes a multi-master serial single-ended busfor communication. The co-processor 230 may receive raw data collected,measured, or detected by each of the IMU 202, the GS camera 208, and theRS camera 212 over the same I2C bus or a separate communication bus. Theco-processor 230 may communicate with the application processor 232 overa number of communication buses including a serial peripheral interface(SPI) bus that includes a synchronous serial data link that may operatein full duplex mode, the I2C bus, and a mobile industry processorinterface (MIPI) that includes a serial interface configured forcommunicating camera or pixel information. Use of various buses may bedetermined based on need of speed of communication of data as well asbandwidth provided by the respective communication bus, for example.

FIG. 4 is a block diagram of an example method 400 for generating adigital image. Method 400 shown in FIG. 4 presents an embodiment of amethod that could be used or implemented by the computing device 100 ofFIG. 1 or the computing device 200 of FIG. 2, for example, or moregenerally by one or more components of any computing device. Method 400may include one or more operations, functions, or actions as illustratedby one or more of blocks 402-408. Although the blocks are illustrated ina sequential order, these blocks may also be performed in parallel,and/or in a different order than those described herein. Also, thevarious blocks may be combined into fewer blocks, divided intoadditional blocks, and/or removed based upon the desired implementation.

In addition, for the method 400 and other processes and methodsdisclosed herein, the block diagram shows functionality and operation ofone possible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor orcomputing device for implementing specific logical functions or steps inthe process. The program code may be stored on any type ofcomputer-readable medium, for example, such as a storage deviceincluding a disk or hard drive. The computer-readable medium may includenon-transitory computer-readable medium, for example, such ascomputer-readable media that stores data for short periods of time likeregister memory, processor cache and random access memory (RAM). Thecomputer-readable medium may also include non-transitory media, such assecondary or persistent long term storage, like read only memory (ROM),optical or magnetic disks, compact-disc read only memory (CD-ROM), forexample. The computer-readable media may also be any other volatile ornon-volatile storage systems. The computer-readable medium may beconsidered a computer-readable storage medium, for example, or atangible storage device.

In addition, for the method 400 and other processes and methodsdisclosed herein, each block in FIG. 4 may represent circuitry that iswired to perform the specific logical functions in the process.

In one embodiment, functions of the method 400 may performed by aco-processor of a mobile device, such as the co-processor 230 of FIG. 2.In other embodiments, the functions of the method 400 may be distributedacross multiple processors that are configured to acquire sensor data,generate a digital image, and/or generate a portion of the digitalimage, such as a particular segment of pixels of the digital image.

Initially, at block 402, the method 400 includes receiving image datafrom at least one camera of the mobile device and sensor data from atleast one sensor of a plurality of sensors of the mobile device. Theimage data may include one or more images received from one or more ofthe global shutter camera 208, rolling shutter camera 210, camera 212,and/or depth processor 228. Thus, in some examples, the image data fromthe at least one camera may include first image data from a first cameraand second image data from a second camera. Furthermore, the image datamay include one or any combination of two-dimensional images andthree-dimensional (e.g., depth) images.

The sensor data my include data from any of the sensors as describedabove in any of FIG. 1, FIG. 2, or FIGS. 3A-3B, for example, includingan IMU, an accelerometer, a gyroscope, a barometer, a magnetometer, anda temperature sensor. It is contemplated that the sensors may includeother types of sensors as well.

At block 404, the method 400 includes generating a digital image thatincludes at least the image data and the sensor data. For example, afile may be generated in a digital image format that includes both theimage data and the sensor data. The image data may already be in pixelformat, therefore the image data may be included in the digital image ina straightforward manner. Optionally, the image data may be convertedfrom a first image format to a second image format in which the digitalimage is generated.

The sensor data may be included within the digital image by embeddingthe sensor data in multiple pixels of the digital image. For instance,the co-processor may be configured to represent the sensor data as fakepixels in the digital image (i.e., pixels whose values have been definedby a processor to represent a portion of a sensor value).

By way of example, the digital image may include pixels that are definedin a brightness and chrominance (YUV) color space. A 16-bit raw sensorvalue can then be embedded in a “Y” space of two pixels. For instance,eight bits of the 16-bit raw sensor value can be represented in a “Y”space of a first pixel and the remaining eight bits can be embedded in a“Y” space of a second pixel. Values in the “U” space and the “V” spaceof the pixels can be set to zero in order to ensure that the sensorvalue is not skewed during image compression or packing, for example.

As another example, a 32-bit single-precision floating point number canbe represented in the “Y” space of four separate 8-bit pixels, whilevalues in the “U” space and the “Y” space can be set to zero. Otherexamples are also possible depending on the number of bytes per pixeland/or color space utilized by the digital image. For instance, thesensor data may be embedded in the “U” space or “V” space instead of the“Y” space.

At block 406, the method 400 includes embedding a frame identifiercorresponding to the digital image in at least two corner pixels ofrespective corners of the digital image. As discussed above, cornerpixels of the digital image may include pixels of the digital image thatare within a threshold number of rows and a threshold number of columnsfrom each corner of the digital image. The frame identifier may be anumber, such as an integer that is incremented sequentially with eachnew digital image that is generated. For instance, for a first digitalimage, the frame identifier may be the integer one, for a second digitalimage, the frame identifier may be the integer two, and so forth.

In some examples, the frame identifier may be embedded in four cornerpixels of four respective corners of the digital image. However, otherconfigurations are also possible, such as embedding the global frameidentifier in corner pixels of two or three respective corners of thedigital image. Depending on the size of the frame identifier, the frameidentifier may be embedded in one or more pixels. For instance, an 8-bitnumber may be embedded within the “Y” space of a single pixel.Similarly, a 16-bit number may be embedded within the “Y” space of twopixels.

At block 408, the method 400 includes providing the digital image to anapplication processor. For instance, the digital image may be providedby the co-processor and to the application processor using a camera businterface, such as a MIPI. In some examples, the digital image may begenerated at 30 Hz, such that a new digital image with new image dataand/or sensor data is provided to the application processor every 33 ms.

In some examples, the method 400 may further include embedding frameidentifiers within corners of one or more segments of pixels within thedigital image. For example, the digital image may include a firstsegment of pixels corresponding to the image data and a second segmentof pixels corresponding to the sensor data. Depending on the desiredconfiguration, a frame identifier may then be embedded within at leasttwo respective corners of the first segment of pixels, and/or a frameidentifier may be embedded within at least two respective corners of thesecond segment of pixels. If the digital image includes more than twosegments of pixels, a frame identifier may similarly be embedded withinany of the additional segments of pixels.

As described herein, a corner pixel of a segment of pixels of thedigital image need not be the precise corner pixel of the segment ofpixels. Rather, a corner pixel of a segment of pixels may refer to apixel that is within a threshold number of columns and a thresholdnumber of rows (e.g., three columns and two rows) from a corner of thesegment of pixels.

For purposes of explanation, the frame identifiers embedded within thecorners of the digital image may be referred to as global frameidentifiers and the frame identifiers embedded within corners of thesegments of the digital image may be referred to as segment frameidentifiers.

In some embodiments, the same frame identifier may be used as the globalframe identifier and the segment frame identifier(s) for a particulardigital image. For instance, the global frame identifier and the segmentframe identifier(s) for a particular digital image may be an integerthat identifies the particular digital image.

In other embodiments, the global frame identifier may be different fromthe segment frame identifier. For instance, if the digital imageincludes two-dimensional image data that is received at 30 Hz andthree-dimensional image data that is received at 15 Hz, and theco-processor is configured to generate digital images at 30 Hz, a firstdigital image and a subsequent digital image may include the samethree-dimensional data. Consequently, the segment frame identifier forthe three-dimensional image data may be different from the global frameidentifier.

Turning now to FIG. 5, FIG. 5 is a conceptual illustration of an exampledigital image format. The conceptual illustration 500 is shown forpurposes of example only, and is not meant to be limiting in any manner.Other configurations and formats for the digital image are alsocontemplated.

As shown in FIG. 5, the example digital image format includes sensordata 502, computer-vision data 504, first camera data 506, depth data508, and second camera data 510. The sensor data 502 is shown within thefirst sixteen rows of a digital image. Thereafter, in the next few rows,the computer-vision data 504 is provided. Consequently, in an example inwhich the digital image is provided to the application processor row byrow, the sensor data 502 would first be provided, followed by thecomputer-vision data 504.

The computer-vision data may identify locations and/or descriptions ofimage features determined using a two-dimensional image and/or imagefeatures determined using depth data. Example types of image featuresinclude edges, corners/interest points, and blobs. An edge is a point inan image where there is a boundary between two image regions. Corners,or more generally interest points, refer to point-like features in animage which have a local two-dimensional structure. Algorithms such asthe Harris & Stephens algorithm can be used to detect edges and corners.The Features from Accelerated Segment Test (FAST) algorithm is anotherexample of an algorithm that may be used to detect corners and interestpoints. Finally, blobs describe regions of interest in an image (e.g.,regions that are too smooth to be detected by a corner detector). TheFAST algorithm, among others, can also be used to detect blobs.

Below the sensor data 502 and computer-vision data 504, which are shownas occupying 31 rows in the example digital image format 500, largerdata sources are provided. The larger data sources include the firstcamera data 506, followed by the depth data 508, and the second cameradata 510. In one example, the first camera data 506 may be an image froma first camera having a field of view that is greater than 120 degrees,such as the GS camera 208 of FIG. 2. Further, the second camera data maybe an image from a second camera having a smaller field of view, such asthe RS camera 210 of FIG. 2.

Note that the example digital image format shown in FIG. 5 is just oneexample. In other instances, other digital image formats may beutilized. For instance, the first camera data 506 and the second cameradata 510 may be located side-by-side within rows 31-510 of a digitalimage (not shown). Similarly, in other example digital image formats,more or less data may be provided. For instance, in another digitalimage format the depth data or the computer-vision data may be omitted,or third camera data from a third camera may also be included. Asanother example, in another digital image format, a combined color anddepth image may be provided in addition to or in place of the firstcamera data 506, the depth data 508, and/or second camera data 510.

In another example digital image format, the digital image may includepadding between one or more of the different data segments. For example,the digital image may be padded with blank or empty data in between thefirst camera data 506 and the depth data 508 or between the depth data508 and the second camera data 510. In some instances, the paddingbetween the different data segments may be inserted in order to makesure that the beginning of each data segment corresponds to thebeginning of a sector or subdivision of a track in a memory.

As an example, padding may be inserted between data segments to makesure that the beginning of the first camera data 506 corresponds to thebeginning of a 4096-byte (4K) sector. This may allow the first cameradata 506 to be written more efficiently to memory. Similarly, thedigital image may be padded such that the beginning of each of the depthdata 508 and second camera data 510 corresponds to the beginning of a 4Ksector.

In another example digital image format, the width of the digital imagemay be 1280 pixels. As a result, each row of the digital image mayinclude two rows of pixel data from the first camera data. The rows maybe organized side-by-side. For example, columns 0-639 of row 31 of thedigital image may include a first row of pixels of the first camera data506 and columns 640-1279 of row 31 may include a second row of pixels ofthe first camera data 506. Similarly, a single row of the digital imagemay include two rows of depth data 508 or two rows of pixels of secondcamera data 510.

In FIG. 6, a conceptual illustration 600 of example sensor data formatsis provided. As shown in FIG. 6, a global frame identifier may beprovided in the “Y” space of first pixel of a first line (e.g., a firstrow of pixels) of a digital image. Additionally, the global frameidentifier may be provided within the “Y” space of the last pixel ofline zero. Furthermore, a segment frame identifier may be embeddedwithin the “Y” space of a second pixel of line 0.

Accelerometer data is provided in line zero as well. In particular, theeight highest bits of a 16-bit x-axis acceleration value (i.e., a highbyte) are shown within the “Y” space of a third pixel while the eightlowest bits of the 16-bit x-axis acceleration value (i.e., a low byte)are shown within the “Y” space of a fourth pixel. Similarly, y-axisacceleration values may be embedded within the “Y” space of pixels 4-5and 6-7 respectively. Furthermore, gyroscope data and magnetometer dataare also embedded within the “Y” space of pixels 0-5 of lines 1 and 2.

The barometer reading is shown as four separate bytes (B3, B2, B1, andB0) that, when combined, represent a 32-bit number. In line 4, aninteger component of a temperature reading is shown in the “Y” space ofpixels 0-1, while a fraction component of the temperature reading isshown in the “Y” space of pixels 2-3. Additionally, a timestamp that hasbeen divided into four eight bit values is also shown in line 5. Inother examples, the sensor data may also include timestamp data for oneor more of the other sensors.

Although sensor data for each type of sensor is located within anindividual row of pixels in FIG. 6, in other examples, the sensor datamay be represented within a single row of pixels (not shown). Forexample, the gyroscope data may be embedded in the “Y” space of pixels8-13, the magnetometer data may be represented in the “Y” space ofpixels 14-19, and so forth.

In line 15, the last line of the segment of pixels corresponding to thesensor data, segment frame identifiers are provided within the “Y” spaceof the first pixel and the last pixel.

In line 16, an example computer-vision data section begins. As shown inFIG. 6, a segment frame identifier is provided within the “Y” space ofthe first pixel and the last pixel of the first line (line 16) of thesegment. Additionally, a first four-byte image feature is embedded inthe “Y” space of pixels 1-4 of line 16. Thereafter, a second, two-byte,image feature may be embedded in the “Y” space of pixels 5-6 (notshown). The segment frame identifier may also be provided within the “Y”space of the first pixel and the last pixel of a last line of thesegment (not shown). Other configurations are also contemplated. Forinstance, in another configuration, the computer-vision data and thesensor data may be provided within a single segment of pixels.

Referring now to FIGS. 7 and 8, conceptual illustrations of exampledigital image formats that include frame identifier(s) are described. Asshown in the conceptual illustration 700 of FIG. 7, in one exampleconfiguration, a global frame identifier 702 may be embedded withincorner pixels of four corners of the digital image. For example, theframe number “2” may be embedded within a corner pixel in each corner ofthe digital image.

FIG. 8 illustrates a conceptual illustration 800 of an example digitalimage format that includes both a global frame identifier 802 anddifferent segment frame identifiers 804, 806. The global frameidentifier 802 is illustrated using a square with a solid-lineperimeter, while the segment frame identifiers 804, 806 are illustratedusing a dotted-line perimeter.

As shown in FIG. 8, the global frame identifier 802 is provided withineach of four corners of the digital image. A first segment identifier804 is provided within four corners of the sensor data segment,computer-vision data segment, first camera data segment, and secondcamera data segment. Furthermore, a second segment identifier 806, thatis different from the first segment identifier 804 and the global frameidentifier 802, is provided within the depth data segment.

As discussed above, an application processor (or other entity) thatreceives a digital frame from the co-processor may be configured to testthe received digital image for errors using the embedded global frameidentifiers and/or segment frame identifiers. FIG. 9 is a block diagramof an example method 900 for testing a digital image. Method 900 shownin FIG. 9 presents an embodiment of a method that could be used orimplemented by the computing device 100 of FIG. 1 or the computingdevice 200 of FIG. 2, for example, or more generally by one or morecomponents of any computing device. Method 900 may include one or moreoperations, functions, or actions as illustrated by one or more ofblocks 902-910. Although the blocks are illustrated in a sequentialorder, these blocks may also be performed in parallel, and/or in adifferent order than those described herein. Also, the various blocksmay be combined into fewer blocks, divided into additional blocks,and/or removed based upon the desired implementation.

In addition, for the method 900, each block may represent a module, asegment, or a portion of program code, which includes one or moreinstructions executable by a processor or computing device forimplementing specific logical functions or steps in the process. Inaddition, each block in FIG. 9 may represent circuitry that is wired toperform the specific logical functions in the process.

In one embodiment, functions of the method 900 may performed by anapplication processor of a mobile device, such as the applicationprocessor 232 of FIG. 2. In other embodiments, the functions of themethod 900 may be distributed across multiple processors of a mobiledevice, or performed by one or more processors of a server that isremote from the mobile device.

Initially, at block 902, the method 900 includes receiving a digitalimage that includes: image data from at least one camera of the a mobiledevice, sensor data from at least one sensor of the mobile device, and aframe identifier embedded in at least two corner pixels of respectivecorners of the digital image. The digital image may be received from aco-processor of the mobile device that is configured to generate thedigital image, as described above with respect to the example method 400of FIG. 4, for example. Thus, the sensor data and the frame identifiermay be embedded within pixels of the digital image.

At block 904, the method 900 includes determining a first value embeddedin a first corner pixel of the digital image and a second value embeddedin a second corner pixel of the digital image. In one example, theapplication processor may be configured to determine values withinpredetermined pixels of the digital image. For instance, the applicationprocessor may be configured to determine the first value embedded in the“Y” space of a pixel in row 1, column 1 of the digital image and asecond value embedded in the “Y” space of a pixel in the last row andlast column of the digital image. Thus, the first value and the secondvalue may be determined by identifying a numerical value embedded in the“Y” space of two or more predetermined pixels of the digital image.

At block 906, the method 900 includes comparing the first value and thesecond value. If the first value is equal to the second value, at block908, the method includes providing an output indicating that the digitalframe is valid. For instance, if the first value and the second valueagree, this may be indicative that a frame tear did not occur.

On the other hand, if the first value is not equal to the second value,at block 910, the method includes providing an output indicating thatthe digital frame is invalid. If the first value and the second value donot agree, the digital frame may have errors, due to a frame tear orother reason. Therefore, the output provided at block 910 may indicatethat the data of the digital frame may have been corrupted during thegeneration or transmission of the digital image. In some examples, inresponse to detecting that the frame identifiers embedded within thedigital image do not agree, the application processor may request thatthe co-processor regenerate and/or resend the digital image.

In one embodiment, a frame identifier may have been embedded in fourcorner pixels of four respective corners of the digital image. Testingthe digital image may then involve determining a first value, secondvalue, third value, and fourth value embedded in a first corner pixel,second corner pixel, third corner pixel, and fourth corner pixelrespectively of the four corner pixels of the digital image. Theapplication processor may then compare the four determined values. Basedon the comparison, the application processor may provide an outputindicative of a validity of the digital image. For instance, in responseto determining that the four values are equal, the application processormay provide an output indicating that the digital image is valid. Asanother example, in response to determining that the four values are notall the same, the application processor may provide an output indicatingthat the digital image is invalid.

In another embodiment, a segment frame identifier may have been embeddedwithin at least two corners of a segment of pixels of the digital image.The application processor may then be configured to test the validity ofthe digital image using the segment frame identifier. FIG. 10 is a blockdiagram of additional or optional functions that may be performed inconjunction with the example method of FIG. 9.

The function at block 1002 involves determining a first value embeddedin a first corner pixel of a segment of pixels of the digital image, anddetermining a second value embedded in a second corner pixel of thesegment of pixels of the digital image. For instance, the applicationprocessor may be configured to determine a first value embedded in the“Y” space of a first predetermined pixel of the segment of pixels and asecond value embedded in the “Y” space of a second predetermined pixelof the segment of pixels.

The function at block 1004 involves comparing the first value and thesecond value. If the first value is equal to the second value, at block1006, an output may be provided indicating that the digital frame isvalid. For example, if the application processor has already determinedthat the embedded global frame identifiers are equal (at block 906), theresult of the comparison at block 1004 may be used to further validatethe data of the digital image.

In another instance, the output provided at block 1006 may further orinstead indicate that the segment of pixels is valid. For example, theoutput provided at block 1006 may indicate that the segment of pixels isvalid, without providing any indication of whether the entire digitalimage is valid or invalid.

On the other hand, if the first value is not equal to the second value,at block 1008, an output may be provided indicating that the digitalframe is invalid. In some instances, the output provided at block 1008may further or instead indicate that the segment of pixels is invalid.In response to determining that a particular segment of pixels isinvalid, the application processor may request that the co-processorregenerate and/or resend the particular segment of pixels.

In other embodiments, a segment frame identifier(s) embedded withinmultiple segments of pixels of the digital image may be tested using thefunctions described with respect to blocks 1002-1008. Testing segmentframe identifier(s) embedded within particular segments of the digitalimage may enable the application processor to identify particularsegments of pixels within the digital image as valid or invalid byproviding multiple outputs.

It should be understood that arrangements described herein are forpurposes of example only. As such, those skilled in the art willappreciate that other arrangements and other elements (e.g. machines,interfaces, functions, orders, and groupings of functions, etc.) can beused instead, and some elements may be omitted altogether according tothe desired results. Further, many of the elements that are describedare functional entities that may be implemented as discrete ordistributed components or in conjunction with other components, in anysuitable combination and location.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopebeing indicated by the following claims, along with the full scope ofequivalents to which such claims are entitled. It is also to beunderstood that the terminology used herein is for the purpose ofdescribing particular embodiments only, and is not intended to belimiting.

What is claimed is:
 1. A method comprising: receiving, by a processor ofa mobile device, image data from at least one camera of the mobiledevice and sensor data from at least one sensor of a plurality ofsensors of the mobile device; generating, by the processor, a digitalimage that includes at least the image data and the sensor data, whereinthe sensor data is embedded in pixels of the digital image; embedding,by the processor, a frame identifier corresponding to the digital imagein at least two corner pixels of respective corners of the digitalimage, wherein the digital image comprises a first segment of pixelscorresponding to the image data and a second segment of pixelscorresponding to the sensor data and wherein embedding the frameidentifier comprises embedding the frame identifier in pixels located inat least two corner pixels of respective corners of the first segment ofpixels and at least two corner pixels of respective corners of thesecond segment of pixels; and providing, by the processor, the digitalimage to an application processor of the mobile device.
 2. The method ofclaim 1, wherein the frame identifier comprises a frame number.
 3. Themethod of claim 1, wherein the at least two corner pixels of therespective corners of the digital image comprise pixels of the digitalimage that are located within a threshold number of columns and athreshold number of rows from the respective corners of the digitalimage.
 4. The method of claim 1, further comprising embedding the frameidentifier corresponding to the digital image in four corner pixels offour respective corners of the digital image.
 5. The method of claim 1,further comprising embedding the frame identifier in pixels located infour corners pixels of four respective corners of the first segment ofpixels and four corner pixels of four respective corners of the secondsegment of pixels.
 6. The method of claim 5, further comprising:receiving depth data from a depth processor of the mobile device,wherein the digital image further comprises a third segment of pixelscorresponding to the depth data; and embedding the frame identifier inpixels located in four corner pixels of four respective corners of thethird segment of pixels.
 7. The method of claim 1, wherein the sensordata comprises data from an inertial measurement unit of the mobiledevice.
 8. A method comprising: receiving, by a processor of a mobiledevice, image data from at least one camera of the mobile device andsensor data from at least one sensor of a plurality of sensors of themobile device, the image data comprising first image data from a firstcamera and second image data from a second camera; generating, by theprocessor, a digital image that includes at least the image data and thesensor data, wherein the sensor data is embedded in pixels of thedigital image; embedding, by the processor, a frame identifiercorresponding to the digital image in at least tow corner pixels ofrespective corners of the digital image; and providing, by theprocessor, the digital image to an application processor of the mobiledevice; wherein the digital image comprises a first segment of pixelscorresponding to the first image data, a second segment of pixelscorresponding to the second image data, and a third segment of pixelscorresponding to the sensor data; and wherein embedding the frameidentifier comprises embedding the frame identifier in pixels located inat least two corner pixels of respective corners of the first segment ofpixels and at least two corner pixels of respective corners of thesecond segment of pixels.
 9. The method of claim 8, wherein the sensordata comprises data from an inertial measurement unit of the mobiledevice.
 10. The method of claim 8, wherein: the at least two cornerpixels of the respective corners of the first segment of pixels comprisepixels that are located within a threshold number of columns and athreshold number of rows from the respective corners of the firstsegment; and the at least two corner pixels of the respective corners ofthe second segment of pixels comprise pixels that are located within athreshold number of columns and a threshold number of rows from therespective corners of the second segment.
 11. A mobile devicecomprising: at least one camera; a sensor; and a co-processor, theco-processor configured to: generate a digital image that includes imagedata from the at least one camera and sensor data from the sensor,wherein the sensor data is embedded in pixels of the digital image, andembed a frame identifier corresponding to the digital image in at leasttwo corner pixels of respective corners of the digital image; and anapplication processor, the application processor configured to: receivethe digital image from the co-processor, determine a first valueembedded in a first corner pixel of the at least two corner pixels and asecond value embedded in a second corner pixel of the at least twocorner pixels, and based on a comparison between the first value and thesecond value, provide an output indicative of a validity of the digitalimage.
 12. The mobile device of claim 11, further comprising a camerabus interface, wherein the application processor is configured toreceive the digital image from the co-processor using the camera businterface.
 13. The mobile device of claim 11, wherein providing theoutput indicative of the validity of the digital image based on thecomparison between the first value and the second value comprisesproviding an output indicating that the digital image is valid when thefirst value and the second value are the same value.
 14. The mobiledevice of claim 11, wherein providing the output indicative of thevalidity of the digital image based on the comparison between the firstvalue and the second value comprises providing an output indicating thatthe digital image is invalid when the first value and the second valueare not the same value.
 15. The mobile device of claim 11: wherein theco-processor is configured to embed the frame identifier correspondingto the digital image in four corner pixels of four respective corners ofthe digital image, and wherein the application processor is configuredto: determine a first value, second value, third value, and fourth valueembedded in a first corner pixel, second corner pixel, third cornerpixel, and fourth corner pixel respectively of the four corner pixels ofthe digital image, and based on a comparison between the first value,second value, third value, and fourth value, provide an outputindicative of a validity of the digital image.
 16. The mobile device ofclaim 11: wherein the image data from the at least one camera comprisesfirst image data from a first camera and second image data from a secondcamera, and wherein the sensor data comprises sensor data from aninertial measurement unit (IMU).
 17. A method comprising: receiving, byan application processor of a mobile device and from a co-processor ofthe mobile device, a digital image that includes image data from atleast one camera of the mobile device and sensor data from at least onesensor of the mobile device, wherein the sensor data is embedded inpixels of the digital image, and wherein the digital image furthercomprises a frame identifier embedded in at least two corner pixels ofrespective corners of the digital image; determining, by the applicationprocessor, a first value embedded in a first corner pixel of the atleast two corner pixels and a second value embedded in a second cornerpixel of the at least two corner pixels of the respective corners of thedigital image; and based on a comparison between the first value and thesecond value, providing an output indicative of a validity of thedigital image.
 18. The method of claim 17: wherein the digital imagecomprises a first segment of pixels corresponding to the image data anda second segment of pixels corresponding to the sensor data, wherein thedigital image comprises a frame identifier embedded in at least twocorner pixels of respective corners of the first segment of pixels, andwherein the method further comprises: determining a third value embeddedin a first corner pixel of the at least two corner pixels of the firstsegment of pixels and a fourth value embedded in a second corner pixelof the at least two corner pixels of the first segment of pixels; andbased on a comparison between the third value and the fourth value,providing another output indicative of a validity of the image data. 19.The method of claim 18: wherein the digital image comprises a frameidentifier embedded in at least two corner pixels of respective cornersof the second segment of pixels, and wherein the method furthercomprises: determining a fifth value embedded in a first corner pixel ofthe at least two corner pixels of the second segment of pixels and asixth value embedded in a second corner pixel of the at least two cornerpixels of the second segment of pixels; and based on a comparisonbetween the fifth value and the sixth value, providing another outputindicative of a validity of the sensor data.
 20. The method of claim 17,wherein the at least two corner pixels of the respective corners of thedigital image comprise pixels of the digital image that are locatedwithin a threshold number of columns and a threshold number of rows fromthe respective corners of the digital image.